Causality and Validity

class: center, middle, inverse, title-slide

# Causality and Validity
## EDP 612 Week 7
### Dr. Abhik Roy

---

section p {
    margin: auto;
}

section {
    height: 600px;
    width: 60%;
    margin: auto;
    border-radius: 21px;
    background-color: #212121;
}

section p {
    text-align: center;
    font-size: 30px;
    background-color: #212121;
    border-radius: 21px;
    font-family: Roboto Condensed;
    font-style: bold;
    padding: 12px;
    color: #bff4ee;
}

#center {

text-align: center;

}

.center p {
  margin: 0;
  position: absolute;
  top: 50%;
  left: 50%;
  -ms-transform: translate(-50%, -50%);
  transform: translate(-50%, -50%);
}

.center2 {
  margin: 0;
  position: absolute;
  top: 50%;
  left: 50%;
  -ms-transform: translate(-50%, -50%);
  transform: translate(-50%, -50%);
}

.tab {
    display: inline-block;
    margin-left: 40px;
}
</style>

<div>
<style type="text/css">.xaringan-extra-logo {
width: 110px;
height: 128px;
z-index: 0;
background-image: url(/Users/skynet/Documents/WVU/Teaching/GitHub.nosync/edp612/static/img/course_hex.png);
background-size: contain;
background-repeat: no-repeat;
position: absolute;
top:1em;right:1em;
}
</style>
<script>(function () {
  let tries = 0
  function addLogo () {
    if (typeof slideshow === 'undefined') {
      tries += 1
      if (tries < 10) {
        setTimeout(addLogo, 100)
      }
    } else {
      document.querySelectorAll('.remark-slide-content:not(.title-slide):not(.inverse):not(.hide_logo)')
        .forEach(function (slide) {
          const logo = document.createElement('a')
          logo.classList = 'xaringan-extra-logo'
          logo.href = 'https://edp612.asocialdatascientist.com'
          slide.appendChild(logo)
        })
    }
  }
  document.addEventListener('DOMContentLoaded', addLogo)
})()</script>
</div>

---

class: highlight-last-item
layout: true

---

#  Experiments and Causation

---

#  Cause

+ Variable that produces an effect or result

+ Most causes are **inus** - 
	
> A cause is an insufficient (**i**)

> but non-redundant (**n**)

> part of an unnecessary (**u**) but

> sufficient condition (**s**)

- A given event may have many different causes
	
--

- Many factors are required for an effect to occur, but they can rarely be fully known and how they relate to one another

---

#  Effect

+ Difference between what did happen and what would have happened

+ This reasoning generally requires a counterfactual
	
---

#  Counterfactual

+ Knowledge of what would have happened in the absence of a suspected causal agent

+ Physically impossible

+ Impossible to simultaneously receive and not receive a treatment
	
--

+ Therefore, the central task of all cause-probing research is to approximate the physically impossible counterfactual

---

#  Causal Relationships

A causal relationship requires three conditions

1. Cause preceded effect (temporal precedence)

2. Cause and effect covary

3. No other plausible alternative explanations can account for a causal relationship

---

#  Cause, Effect, and Causal Relationships

+ In experiments

+ Presumed causes are manipulated to observe their effect

+ Variability in cause related to variation in an effect

+ Elements of design and extra-study knowledge are used to account for and reduce the plausibility of alternative explanations

---

#  Causation, Correlation, and Confounds

+ Correlation does not prove causation

+ Correlations do not meet the first premise of causal logic (temporal precedence)

+ Such relationships are often due to a third variable (i.e., a confound)

---

#  Manipulable and Nonmanipulable Causes

+ Experiments involve causal agents that can be manipulated

+ Nonmanipulable causes (e.g., ethnicity, gender) cannot be causes in experiments because they cannot be deliberately varied

---

#  Causal Description and Causal Explanation

+ **Causal description**. identifying that a causal relationship exists between A and B

+ **Molar causation**. the overall relationship between a treatment package and its effects

+ **Causal explanation**. explaining how A causes B

+ **Molecular causation**. knowing which parts of a treatment are responsible for which parts of an effect

---

#  Causal Models

---

#  Causal Models

---

#  Modern Descriptions of Experiments

---

#  Randomized Experiment

+ Units are assigned to conditions randomly

+ Randomly assigned units are probabilistically equivalent based on expectancy (if certain conditions are met)
	
--

+ Under the appropriate conditions, randomized experiments provide unbiased estimates of an effect

---

#  Quasi-Experiment

+ Shares all features of randomized experiments except assignment

+ Assignment to conditions occurs by self-selection

+ Greater emphasis on enumerating and ruling out alternative explanations

+  ... through logic and reasoning, design, and measurement

---

#  Natural Experiment

+ Naturally-occurring contrast between a treatment and comparison condition

+ Typically concern nonmanipulable causes

+ Requires constructing a counterfactual rather than manipulating one

---

#  Nonexperimental Designs

+ Often called correlational or passive designs (i.e., cross-sectional)

+ Statistical controls often used in place of structural design elements

+ Generally do not support strong causal inferences

---

#  Experiments and the Generalization of Causal Connections

---

#  Most Experiments are Local but have General Aspirations

+ Most experiments are localized

+ Limited samples of **utos**

> units (**u**)

> treatments (**t**)

> observations (**o**)

> settings (**s**)

+ What Campbell labeled local molar causal validity

---

#  Construct Validity: Causal Generalization as Representation

+ *Premised on generalizing from particular sampled instances of units, treatments, observations, and settings to the abstract, higher order constructs that sampled instances represent*

---

#  External Validity: Causal Generalization as Extrapolation

+ *Inferring a causal relationship to unsampled units, treatments, observations, and settings from sampled instances*

+ Enhanced when probability sampling methods are used

+ Broad to narrow

+ Narrow to broad

---

#  Approaches to Making Causal Generalizations

+ Sampling

+ Probabilistic

+ Heterogeneous instances

+ Purposive

+ Grounded theory

+ Surface similarity

+ Ruling out irrelevancies

+ Making discrimination

+ Interpolation and extrapolation

+ Casual explanation

---

# Statistical Conclusion Validity and Internal Validity

---

#  Validity

+ Approximate truthfulness of correctness of an inference

+ Not an all or none, either or, condition, rather a matter of degree

+ Efforts to increase one type of validity often reduce others

---

#  Statistical Conclusion Validity

Validity of inferences about the covariation between treatment (cause) and outcome (effect)

---

#  Internal Validity

--
 
*Validity of inferences about whether observed covariation between `\(A\)` (treatment/cause) and `\(B\)` (outcome/effect) reflects a causal relationship from `\(A\)` to `\(B\)` as those variables were manipulated or measured*

---

#  Construct Validity

*Validity of inferences about the higher order constructs that represent sampling particulars*

---

#  External Validity

*Validity of inferences about whether a cause-effect relationship holds over variations in units, treatments, observations, and settings*

---

#  Threats to Validity

+ Reasons why an inference may be partly or wholly incorrect

+ Design controls can be used to reduce many validity threats, but not in all instances

+ Threats to validity are generally context-dependent

---

#  Internal Validity

+ *Inferences about whether the observed covariation between `\(A\)` and `\(B\)` reflects a causal relationship from `\(A\)` to `\(B\)` in the form in which the variables were manipulated or measured*

+ In most cause-probing studies, internal validity is the primary focus

---

#  Threats to Internal Validity (1/2)

+ **Ambiguous temporal precedence**. Lack of clarity about which variable occurred first may yield confusion about which variable is the cause and which is the effect

+ **Selection**. Systematic differences over conditions in respondent characteristics that could also cause the observed effect

+ **History**. Events occurring concurrently with treatment that could cause the observed effect

+ **Maturation**. Naturally occurring changes over time that could be confused with a treatment effect

+ **Regression**. When units are selected for their extreme scores, they will often have less extreme scores on other variables, an occurrence that can be confused with a treatment effect

---

#  Threats to Internal Validity (2/2)

+ **Attrition**. Loss of respondents to treatment or measurement can produce counterfactual effects if that loss is systematically correlated with conditions

+ **Testing**. Exposure to a test can affect test scores on subsequent exposures to that test, an occurrence that can be confused with a treatment effect

+ **Instrumentation**. The nature of a measure may change over time or conditions in a way that could be confused with a treatment effect

+ **Additive and interactive threats**. The impact of a threat can be added to that of another threat or may depend on the level of another threat

---

#  Estimating Internal Validity in Experiments

+ By definition randomized experiments eliminate selection through random assignment to conditions

+ Most other threats are (should be) probabilistically distributed as well

---

#  Estimating Internal Validity in Experiments

+ Only two likely validity threats (typically) arise from experiments

+ Attrition

+ Testing

---

#  Estimating Internal Validity in Quasi-Experiments

+ Differences between groups tend to be more systematic than random

+ All threats should be made explicit and then ruled out one by one
	
--

+ Once identified, threats can be systematically examined

---

#  That’s it!

Any questions?